Logical inconsistencies in time trade-off valuation of EQ-5D-5L health states: Whose fault is it?

INTRODUCTION Inconsistency in the time trade-off (TTO) task in EQ-5D-5L occurs when a respondent gives a higher value to a logically worse health state, the occurrence of inconsistency compromises the quality of the data. It is not yet clear which factors are associated with individual level inconsistency. Relating inconsistency to the characteristics of the respondent, interviewer, and the interview process could be helpful in understanding the causes of inconsistency. The objective of this paper is to discover the factors associated with individual level inconsistencies. METHODS Twenty interviewers interviewed 1,296 respondents and each respondent valued 10 health states using the EQ-VT platform in 5 cities in China. At the respondent level, inconsistency was identified in terms of severity and quantity and related to the respondent's background characteristics, the time and iterations spent on the wheelchair example task, and the formal TTO tasks, using multilevel multinomial regression analyses. Interviewers' impact on inconsistencies was analyzed using single level multinomial regression analyses. RESULTS In the full dataset, slight inconsistency was more related to the interview process (Time spent on TTO task: RRR = 1.246 with 95%CI: 1.076,1.441; time spent on Wheelchair example: RRR = 0.815 with 95%CI:0.699,0.952) while severe inconsistency was more related to respondent's gender (Gender: RRR = 2.347 with 95%CI:1.429,3.855). One Interviewer (Interviewer 7: RRR = 7.335 with 95%CI:1.908,28.195) and interviewer's experience (Sequence: RRR = 0.511 with 95%CI:0.385,0.678) in general showed strong influence over inconsistency in the TTO task. CONCLUSION In conclusion, logical inconsistency in the valuation of EQ-5D-5L health states is associated not only with respondents' characteristics but also with interviewers' performance and the interview process. The role of interviewers and the importance of interviewer training may be more crucial than hitherto believed. This finding could be generalizable to other interviewer-administered health-state valuation study.